How to improve the accuracy of recommendation is an important issue for the current recommendation system. The matrix decomposition model was studied, and in order to exploit the group structure of the rating data, a Score Similarity based Matrix Factorization recommendation algorithm with Group Sparsity (SSMF-GS) was proposed. Firstly, the scoring matrix was divided into groups according to the users' rating behavior, and the similar user group scoring matrix was obtained. Then, similar users' rating matrix was decomposed in group sparsity by SSMF-GS algorithm. Finally, the alternating optimization algorithm was applied to optimize the proposed model. The latent item features of different user groups could be filtered out and the explanability of latent features was enhanced by the proposed model. Simulation experiments were tested on MovieLens datasets provided by GroupLens website. The experimental results show that the proposed algorithm can improve recommendation accuracy significantly, and the Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) both have good performance.